Hunting M365 Invaders: Navigating the Shadows of Midnight Blizzard

This blog is an independent publication and is neither affiliated with, nor authorized, sponsored, or approved by, Microsoft Corporation.

On January 19, Microsoft issued an advisory disclosing a cybersecurity incident targeting their M365 tenants and attributing the attack to Midnight Blizzard, a state-sponsored actor also known as Nobelium and APT29. Following this, on January 24, the Microsoft team expanded on the initial announcement with a comprehensive blog post providing more insights about the attack and outlining specific tactics, techniques and procedures leveraged by the threat actor. Additionally, security researcher Andy Robbins contributed valuable insights with a blog post and a video serving as a key resource for defenders seeking to better understand this incident.

In this blog post, the Splunk Threat Research Team walks through the attack chain described by the Microsoft blog post aiming to identify and share practical detection and hunting strategies for cybersecurity defenders. Recognizing that some of the attack specifics remain unknown, we will base our analysis on informed assumptions and also outline broader detection strategies which, while not specific to this incident, can be applied in similar scenarios.

Decoding the Attack Chain

In this section, we'll dissect the attack chain followed by Midnight Blizzard, breaking it down tactic by tactic. For each tactic, we begin with a key statement from Microsoft's blog post describing the steps taken by the adversary and explore them for potential detection opportunities.

The detection strategies presented are primarily derived from Office 365's Unified Audit Log and have been detailed in the Nobelium Group analytic on our research site. They can also be found across the Office 365 stories: Office 365 Account Takeover, Office 365 Persistence Mechanisms and Office 365 Collection Techniques. For those preferring to use Entra ID logs as their primary data source, corresponding Entra ID detections can also be found there: Azure AD Account Takeover, Azure AD Persistence and Azure AD Persistence.

Finally, our attack_data open source project contains the datasets from all the simulated attack techniques in this post. It's a key resource for detection engineers, especially those unable to conduct their own simulations, providing practical data to develop and validate detection strategies.

Initial Access

^Source

Detection engineers can leverage error code 50126 on both the Unified Audit Log and Entra ID logs to identify a traditional password spraying attack where a high number of users fail to authenticate from one single source IP in a short period of time.

O365 Multiple Users Failing To Authenticate From Ip

`o365_management_activity` Workload=AzureActiveDirectory Operation=UserLoginFailed ErrorNumber=50126    
|  bucket span=5m _time  
|  stats dc(user) as unique_accounts values(user) as user values(LogonError) as LogonError 
values(signature) as signature values(UserAgent) as UserAgent by _time, src_ip  
|  where unique_accounts > 10

Adversaries may choose to execute stealthier password spray campaigns that use a distributed network of IP addresses across various countries to evade security controls. This may bypass the previous analytic logic. However, we can use a different strategy: identifying authentication spikes within a short period of time that exhibit specific characteristics.

In the following analytic, we calculate key metrics such as:

• Number of unique user-IP combinations
• Count of distinct users
• Diversity of source IPs and countries
• Unique number of user-agents involved in failed login attempts 

By customizing the thresholds for these metrics, we can hunt for patterns that deviate from normal behavior.

O365 Multi-Source Failed Authentications Spike

`o365_management_activity` Workload=AzureActiveDirectory Operation=UserLoginFailed ErrorNumber=50126  
| bucket span=5m _time   
| eval uniqueIPUserCombo = src_ip . "-" . user  
| stats dc(uniqueIPUserCombo) as uniqueIpUserCombinations, dc(user) as uniqueUsers, 
dc(src_ip) as uniqueIPs, values(user) as user, values(src_ip) as ips, values(user_agent) as user_agents by _time  
| where uniqueIpUserCombinations > 20 AND uniqueUsers > 20 AND uniqueIPs > 20

Credential Access

^Source

To compromise the legacy OAuth application and act on behalf of it, the actor had to alter its configuration and add new credentials to the application registration object. We can monitor for these actions by filtering on the “Update application - Certificates and secrets management event”. By focusing on this event, we can monitor and respond to this common tactic in Entra ID privilege escalation attacks.

O365 Service Principal New Client Credentials

`o365_management_activity` Workload=AzureActiveDirectory Operation="Update 
application*Certificates and secrets management "  
|  stats earliest(_time) as firstTime latest(_time) as lastTime by user 
ModifiedProperties{}.NewValue object ObjectId  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

Once the credentials were added, the adversary had to authenticate as the Service Principal associated with the application registration to utilize its permissions. It's important to note that by default, the Unified Audit Log only logs user interactive authentication events and does not include service principal authentication. To effectively monitor these activities, we need to rely on Entra ID logs and the ServicePrincipalSignInLogs category

Understanding which Service Principals are authenticating and from which locations within your organization can help identify unauthorized access and abuse. Unfortunately, mapping these relationships may be challenging in large environments with numerous applications. Despite the complexity, it's a worthwhile effort for security teams to track and inventory source IPs associated with Service Principals as a threat hunting exercise.

Azure AD Service Principal Authentication

`azure_monitor_aad` operationName="Sign-in activity" category=ServicePrincipalSignInLogs  
| rename properties.* as *   
| stats count earliest(_time) as firstTime latest(_time) as lastTime by user, user_id, src_ip, 
resourceDisplayName, resourceId  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

Privilege Escalation

^Source

The exact nature of the high-level permissions granted to the breached application isn't clear, but it's likely they were established prior to the attack. This is a frequent issue in many organizations where identities are often allocated more permissions than necessary. In the M365/Entra ID ecosystem, monitoring Entra ID roles and API permissions (like Graph and Exchange Online) is crucial.

Monitoring these events can be effectively done through the ‘Add member to role’ and ‘Update application’. Defenders can focus on crucial roles like 'Global Administrator' or 'Privileged Role Administrator' and sensitive API permissions such as 'Application.ReadWrite.All', 'AppRoleAssignment.ReadWrite.All', and 'RoleManagement.ReadWrite.Directory'. This is not an extensive list and should be tailored by defenders.

O365 High Privilege Role Granted

 `o365_management_activity` Operation="Add member to role." 
Workload=AzureActiveDirectory  
| eval role_id = mvindex('ModifiedProperties{}.NewValue',2)  
| eval role_name = mvindex('ModifiedProperties{}.NewValue',1)  
| where role_id IN ("29232cdf-9323-42fd-ade2-1d097af3e4de", 
"f28a1f50-f6e7-4571-818b-6a12f2af6b6c", "62e90394-69f5-4237-9190-012177145e10")   
| stats earliest(_time) as firstTime latest(_time) as lastTime by user Operation ObjectId 
role_name  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

O365 Privileged Graph API Permission Assigned

`o365_management_activity` Workload=AzureActiveDirectory Operation="Update application." 
| eval newvalue = mvindex('ModifiedProperties{}.NewValue',0) 
| ssrc input=newvalue  
| search 
"{}.RequiredAppPermissions{}.EntitlementId"="1bfefb4e-e0b5-418b-a88f-73c46d2cc8e9" OR 
"{}.RequiredAppPermissions{}.EntitlementId"="06b708a9-e830-4db3-a914-8e69da51d44f" 
OR "{}.RequiredAppPermissions{}.EntitlementId"="9e3f62cf-ca93-4989-b6ce-bf83c28f9fe8"  
| eval Permissions = '{}.RequiredAppPermissions{}.EntitlementId' 
| stats count earliest(_time) as firstTime latest(_time) as lastTime values(Permissions) 
by user, object, user_agent, Operation 
| `security_content_ctime(firstTime)` 
| `security_content_ctime(lastTime)`

Persistence

When new OAuth applications are created in an Entra ID environment, 'Add Application' and 'Add Service Principal' events are triggered. Monitoring application creations can be challenging due to frequent legitimate triggers in active environments.

However, scenarios where multiple applications are created in a short period of time may be interesting to monitor. Furthermore, our analysis revealed that the 'Actor' field in the Unified Audit Log can be used to determine whether an application was created by a user or a service principal. This extra insight provides additional context, enabling us to develop targeted detection analytics.

O365 Multiple Service Principals Created by User

 `o365_management_activity`  Workload=AzureActiveDirectory Operation="Add service 
principal."   
| bucket span=10m _time  
| eval len=mvcount('Actor{}.ID')  
| eval userType = mvindex('Actor{}.ID',len-1)  
| search userType = "User"  
| eval displayName = object  
| stats count earliest(_time) as firstTime latest(_time) as lastTime values(displayName) as 
displayName dc(displayName) as unique_apps by src_user  
| where unique_apps > 3  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

O365 Multiple Service Principals Created by SP

`o365_management_activity` Workload=AzureActiveDirectory Operation="Add service principal."  
| bucket span=10m _time 
| eval len=mvcount('Actor{}.ID') 
| eval userType = mvindex('Actor{}.ID',len-1) 
| search userType = "ServicePrincipal" 
| eval displayName = object 
| stats count earliest(_time) as firstTime latest(_time) as lastTime values(displayName) as 
displayName dc(displayName) as unique_apps by src_user 
| where unique_apps > 3 
| `security_content_ctime(firstTime)` 
| `security_content_ctime(lastTime)`

^Source

Midnight Blizzard granted themselves a sensitive Exchange Online API permission to move on to email collection. In a typical high-privileged API permission assignment process via the Azure portal, two critical events occur:

1. First, the permission is assigned to the OAuth application, generating an 'Update application' event.
2. Then, an administrator consents to the permission, triggering a 'Consent to Application' event.

Admin consents that grant organization-wide permissions, known as tenant-wide admin consents, are particularly sensitive. These consents allow applications to act on behalf of the entire organization, making them vital to monitor. Inspecting the ModifiedProperties field allows us to develop an analytic for tenant-wide admin consents.

O365 Tenant Wide Admin Consent Granted

`o365_management_activity` Operation="Consent to application."   
| eval new_field=mvindex('ModifiedProperties{}.NewValue', 4)  
| rex field=new_field "ConsentType: (?[^\,]+)"  
| rex field=new_field "Scope: (?[^\,]+)"   
| search  ConsentType = "AllPrincipals"   
| stats count min(_time) as firstTime max(_time) as lastTime by Operation, user, object, 
ObjectId, ConsentType, Scope  
| `security_content_ctime(firstTime)`   
| `security_content_ctime(lastTime)`

However, the attack chain described by Microsoft likely would not have triggered the 'Update Application' or 'Consent to Application' events. The above statement implies the attackers didn't use the Azure portal for standard permission assignments. Instead, they abused the legacy application's service principal privileges to programmatically alter permissions, bypassing normal consent procedures.

To gain a deeper understanding of this method, we replicated the steps using the New-MgServicePrincipalAppRoleAssignedTo commandlet from the Microsoft Graph PowerShell SDK. This simulation confirmed our theory: only the 'Add app role assignment to service principal' event is triggered, unlike the usual two events. 

$servicePrincipal = Get-MgServicePrincipal -Filter "displayName eq 'MaliciousApp'"
$EolServicePrincipal = Get-MgServicePrincipal -Filter "servicePrincipalType eq 'Application' 
and displayName eq 'Office 365 Exchange Online'"
$appRole = $EolServicePrincipal.AppRoles | Where-Object { $_.Value -eq 
"full_access_as_app" -and $_.AllowedMemberTypes -contains "Application" }

$params = @{ 
principalId = $servicePrincipal.Id 
resourceId = $EolServicePrincipal.Id 
appRoleId = $appRole.Id
}
New-MgServicePrincipalAppRoleAssignedTo -ServicePrincipalId $servicePrincipal.Id 
-BodyParameter $params

When looking into the 'Add app role assignment to service principal' event in the logs, we noticed the ‘Actor’ field can be used to determine if a service principal or an admin user performed the action. This led us to create a new analytic designed to catch cases where service principals might be bypassing the admin consent process by assigning API permissions to other service principals programmatically.

O365 Admin Consent Bypassed by Service Principal

`o365_management_activity` Workload=AzureActiveDirectory Operation="Add app role 
assignment to service principal."  
| eval len=mvcount('Actor{}.ID')  
| eval userType = mvindex('Actor{}.ID',len-1)  
| eval roleId = mvindex('ModifiedProperties{}.NewValue', 0)  
| eval roleValue = mvindex('ModifiedProperties{}.NewValue', 1)  
| eval roleDescription = mvindex('ModifiedProperties{}.NewValue', 2)  
| eval dest_user = mvindex('Target{}.ID', 0)  
| search userType = "ServicePrincipal"  
| eval src_user = user  
| stats count earliest(_time) as firstTime latest(_time) as lastTime by src_user dest_user 
roleId roleValue roleDescription  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

Collection

^Source

With the right permissions secured, the adversary began collecting email details from corporate accounts. Defenders can track such activities using the ‘Mailitemsaccessed’ event in the Unified Audit Log. Initially available only for E5 licenses, this logging is being extended to standard M365 users. 

An important element in this event is the ClientAppId, which identifies the ID of the Microsoft Entra app that performed the access on behalf of the user. During our simulation of this technique leveraging an OAuth application, Exchange Web Services, PowerShell and Python clients, we consistently found the ClientAppId to be 47629505-c2b6-4a80-adb1-9b3a3d233b7b.

However, we have not been able to find Microsoft documentation confirming that this GUID is actually assigned to the Exchange Web Services. To avoid misclassifications, instead, we leveraged the ClientInfoString field, which provides information about the email client that was used to perform the operation. In our testing, this field always started with the same string.

O365 OAuth App Mailbox Access via EWS

`o365_management_activity` Workload=Exchange Operation=MailItemsAccessed AppId=* 
ClientAppId=*  
| regex ClientInfoString="^Client=WebServices;ExchangeWebServices"  
|  stats count earliest(_time) as firstTime latest(_time) as lastTime values(ClientIPAddress) 
as src_ip by user ClientAppId OperationCount AppId ClientInfoString  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

While Midnight Blizzard chose to leverage Exchange Web Services for email collection this time, defenders should be aware that the scenario would have also allowed them to obtain sensitive Microsoft Graph API permissions and collect emails in a different way. Knowing this, we also wrote a different analytic flagging Graph API mailbox access.

O365 OAuth App Mailbox Access via Graph API

`o365_management_activity` Workload=Exchange Operation=MailItemsAccessed AppId=* 
AppId=00000003-0000-0000-c000-000000000000  
|  stats count earliest(_time) as firstTime latest(_time) as lastTime values(ClientIPAddress) 
by user ClientAppId OperationCount AppId  
| `security_content_ctime(firstTime)`  
| `security_content_ctime(lastTime)`

An extra behavior worth monitoring for is when multiple mailboxes are accessed by an OAuth application programmatically via an API such as EWS or Microsoft Graph in a short period of time. While journaling and compliance applications may legitimately require such access, applications with such extensive mailbox access are typically exceptions and should be well-documented and inventoried by security teams.

O365 Multiple Mailboxes Accessed via API

`o365_management_activity` Workload=Exchange Operation=MailItemsAccessed AppId=* 
ClientAppId=*   
| bucket span=10m _time  
| eval matchRegex=if(match(ClientInfoString, 
"^Client=WebServices;ExchangeWebServices"), 1, 0)   
| search (AppId="00000003-0000-0000-c000-000000000000" OR matchRegex=1)  
| stats values(ClientIPAddress) as src_ip dc(user) as unique_mailboxes values(user) as user 
by _time ClientAppId ClientInfoString  
| where unique_mailboxes > 5

Closing Thoughts

In the evolving landscape of cloud computing, organizations are increasingly exposed to novel attack vectors like those we've explored today. Misconfigurations in cloud environments can open the door for attackers to gain extensive control over a tenant. It's crucial for defenders to evolve beyond traditional endpoint-focused strategies and deepen their understanding of these emerging threats in the cloud. 

The Splunk Threat Research Team remains committed to empowering the cybersecurity community through content like detection strategies, SOAR playbooks, blog posts and open source projects. We hope that this blog post serves as a resource for defenders to strengthen their posture against similar nation-state attacks and adapt to the ever-changing cyber threat landscape.

Learn More

Visit research.splunk.com to view the Splunk Threat Research Team's complete security content repository. You can implement this content using the Enterprise Security Content Updates app or the Splunk Security Essentials app.